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ABSTRACT 

The shortcomings of the dominant information 
processing models of cognition are outlined, and two alterative 
models derived from semiotics are presented. In addition, the 
possibility of incorporating J. J. Gibson's ecological theory of 
affordance within the semiotic models is explored as a means of 
addressing some criticism of the latter models. The semiotic models 
addressed are J. Deely's (1983, 1986) sensation-based Umwelt model 
and U. Eco^s (1976, 1979, 1984) Model Q. The criticism that semiotic 
models lead to solipsism is dealt with through Gibson's rejection of 
perception as based solely on sensation. In his theory of visual 
perception, Gibson considers the environment to be the surfaces that 
separate substances from the mediiam in which animals live. But 
environments also "afford'' things, such as shelter and locomotion. 
The processes of perceiving affordances and abduction, as described 
by Deely, allow semiotics to escape solipsism. Five figures are 
provided. (TJH) 
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Donald J. Cunningham 
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In this paper, I would like to briefly outline the 
shortcomings of the dominate information processing models of 
cognition r present two alternative models derived from semiotics 
and, finally, explore the possibility of incorporating J. J. 
Gibson's theory of affordance within the semiotic models to 
address some of the criticisms of them. This is a full agenda for 
a paper of this length and I will apologize in advance for 
skimming over some rather complicated and profound issues. What I 
will be sharing with you today are really matters which I hope to 
be working on for several years to come, not a completed project. 
I invite your reactions. (See also Cunningham, in press) 

Cognition as Information Processing 

Cognitive information processing models of cognition view 
man as a symbol manipulator in the same sense that a computer is; 
that is, both use symbols and perform operations on those 
symbols. Human intellectual competence is conceived to be the 
result of a few elementary operations on a set of symbols. These 
operations, like those of the computer, may be combined, 
organized and reorganized in predictable ways, and underlie the 
incredible range of behavior we observe in the human species. 
Complex skills such as driving an automobile, writing a paper for 
AERA, solving a waste management problem for a city, monitoring 
the radar screen in the air traffic control tower at O'Hare, 
etc., result simply from the operation of these fundamental 
processes on the human symbol system. The central task that 
cognitive psychologists have set for themselves is to uncover the 
nature of these symbols and processes. 

But I for one have serious reservations about this paradigm. 
It, as well as its predecessor, the behavior ist model, assume 
that human intellectual skills can be decomposed into discrete 
sets of simple behaviors, symbols or cognitive processes and that 
higher skills are some concatenation of simpler learnings and/or 
processes. Knowledge or understanding is viewed as some entity 
which can be transferred from one place to another (i.e., from 
the environment to the mind or from one location to another in a 
memory system) . It is as if these symbols completely represent 
the aspects of the environment they stand for and exist in stable 
form like bits of information in a computer waiting to be 
operated upon. 

From my admittedly biased perspective, problems have arisen 
with these views, problems so serious that they are producing 
cracks in the foundations of current information processing 
cognitive psychology. A typical scenario is that a theoretical 
entity is proposed like schema (e.g., Rumelhart and Ortony, 1977) 
which is then championed for a period of time, then revised to 
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accommodate to limitations and discrepancies which have arisen r 
and then finally discarded. To illustrate^ Rumelhart (1980) and 
others have proposed that schemata are the fundamental elements 
on which all information processing depends o They are "data 
structures" within which the knowledge stored in memory is 
represented. Schemata contain "slots*' into which incoming 
knowledge is encoded and stored or from which it is retrieved. 
At the heart of the research literature is the assumption that it 
will be possible to somehow accurately portray these schemata 
(i.e.r The burglar schema , the restaurant .script , the face frame, 
etc.) and even construct computer programs which would simulate 
schema based information processing • 

But as more research is done and as the concepts are applied 
more widely, the cracks become increasingly apparent. Comments 
like "Of course the choice of schema which might be applied 
depends on the context "; or "Of course this particular schema is 
only one of many which must be brought to bear to fully 
understand this experience" or "Even the simplest event has an 
infinite number of features which could be attended to". And so 
forth. The notion that schemata themselves are interrelated in 
an enormous variety of ways has lead some theorists to begin to 
explore concepts like tangled hierarchies, metaschemata , 
spreading activation, executive routines, parallel distributed 
processing, etc. What is overlooked in this theorizing is that 
none of the existing models will ever be able to be revised 
enough to accommodate to the seemingly infinite capacity of 
humans to understand things from a seemingly limitless number of 
perspectives. The interconnectedncss of all knowledge is not an 
obstacle to be overcome by our theoretical models — it is the 
fundamental assumption with which we should start. 

Semiotic Models of Cognition 

Let me illustrate how semiotic models of cognition would 
differ from a cognitive information processing view by briefly 
reviewing John Decly's Umwelt model and Umberto Eco's Model 

A. Deely (1983, 1986) 

Deely's Umwelt model, a term borrowed from the late 19th~ 
early 20th century biologist, Jacob von Uexkull, is shown in 
Figure 1. An advantage of this notion is that it applies equally 
well to humans and alloanimals, and, perhaps also to 
alloorganisms (i.e., plants, microbes, etc.). von Uexkull was 
interested in characterizing how animals picture the world in 
their mind and how they then interact with the world as they have 
circumscribed it. Since animals can only respond to a small 
portion of the total sensory information available, they create, 
both as a species and as individual members of a species, an 
Umwelt, a "subjective environment" which details only those 
aspects of the physical world which are important (i.e., to be 
approached, avoided, ignored, etc.). It is crucial to realize 
the difference between an Umwelt and an environment. An 
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environment is a physical setting that can be conceived of 
independently of any particular organism and, in fact, is usually 
said to exist for all organisms. This separation of organism and^ 
environment is a fundamental tenet of cognitive information 
processing models of cognition. The Umwelt of an organism, 
hovever, is not independent of the organism; in fact, it exists 
only in relation to the organism. Any particular physical entity 
can serve an enormous variety of Umwelts: the Mariott Hotel in 
New Orleans can create a shelter from the rain for humans, a 
nesting site for cockroaches, a landing site for pigeons, a 
landmark for cab drivers, and so forth. In all cases, the 
environment of the hotel is the same; that is, the sheltered 
enclosure, the crevices, the flat surfaces, etc, are available to 
each of the organisms, yet their experience of them is quite 
different , 

Thus far the discussion of Umwelt applies equally well to 
both humans and alloanimals. This state of affairs is depicted 
below the double dashed lines in the drawing by Deely (Figure 1), 
Sensory information in the form that are traditionally referred 
to as proper sensibles (i,e,, colors, sounds, textures, tastes 
and odors) and common sensibles (i.e, plurality, shape, position, 
movement and size) are the raw materials out of which Umwelts 
emerge both for species and individuals. At the level which Deely 
labels perception, the Umwelt of the animal emerges; that is, it 
is both ontologically present and idiographically developed by 
particular activities and mediated by processes of synthesis, 
remembering imagining, evaluating and expressing. Via perception, 
the animal comes to terms with the physical environment, creating 
and living in a world uniquely defined for that species and that 
individual. Yet the Umwelt is not static (i.e., in equilibrium) 
but in a constant state of flux both at the species and 
individual level (see Anderson et al . , 1986), 

Semiosis in humans, while based upon the processes described 
thus far, is qualitatively different from that of alloanimals. 
Humans can create signs which go beyond the immediate experience 
of the cognizing organism. Words, pictures, bodily movements and 
the like generate interpretants for objects which need have no 
basis in the real world and which can be manipulated independent 
of that world. Yet these signs come to form a part of the Umwelt 
of humans in the same way that dark crevices do for an insect. 
It is the intervention of language/ according to Deely, that 
allows humans to engage in this type of semiosis. Through 
language, we create culture: governments, armies, schools^ art, 
professional associations, etc. Culture, in turn impacts our 
Jives, by determining what is important, what makes sense, what 
is to be valued, etc. The arbitrary nature of these signs, their 
lack of true rea.'.ity status, is not readily apparent to humans 
until they are exposed to cultural systems which depart from 
their own. 

The fact that humans can utilize signs which are arbitrary 
and need have no existence in their immediate experience is what 
makes thought possible and distinctly human, Deely' s diagram 



depicts this notion by showing that experience comes to be 
represented by linguistic signs that can be created without any 
actual embodiments in the physical world* But these si^'ns come to 
be part of our Umwelt - note the leftward branching lines that 
return to the perceptual level, indicating that we tend to see 
the world anew once some aspect of culture is created. 

Figures 2 and 3 show the dynamic aspects of Deely's model. 
Essentially this is a model of inference. Semiosis is a process 
of applying signs to understand some phenomena (induction) r 
reasoning from sign to sign (deduction) , and/or inventing signs 
to make sense of some new experience (abduction). The bottom 
diagram is meant to suggest the cyclic nature of this inferencing 
process: signs are invented to account for experience, these 
signs are linked to existing sign structures and then used to 
define the Umwelt for that organism. But the world is not 
infinitely malleable to our sign structures and abductive process 
will be again instigated. Deely is here, in my view, 
incorporating growth into his model, both from the perspective of 
ontology and experience. 

B. Eco (1976, 1979, 1984) 

Umberto Eco's Model Q is best presented by recapitulating 
his critique of existing models of semantic representation in his 
book Theory of Semiotics . Figure 4 is an example of a typical 
model of semantic memory (taken from Katz and Fodor, 1963) in 
which the meaning, concept, or scheme of bachelor is represented 
as a hierarchical tree structure. (Eco could just as easily use 
Collins and Quillian, John Anderson's Act * Theory, Rumelhart's 
Schema Theory, etc.). 

In a brilliant critique of the Katz-Fodor model, Eco raises 
a number of issues, among which are the following: 

1. Thc K~F model represents the ideal competence of an 
ideal speaker and is unable to account for the 
social use of languages with all its living 
contradictions, illogic, creativity and so forth. 
The representation of "bachelor" is what one would 
find in a dictionary where dominate meanings are 
listed. What is needed is a concrete encyclopedia 
which lists common social (cultural) beliefs which 
are sometimes mutually contradictory and always 
historically rooted, rather than undated, 
theoretically fixed constructs. 

2. A concept's meaning is claimed to be fixed by 
reference to a finite set of semantic markers which 
are not subject to further analysis. Thus there 
are meaning "kernels" to which the meaning of any 
given concept can finally be reduced. But in fact 
every m^irker must be interpreted by other markers 
(their intcrpretants ) and given that there are an 
infinite number of potential markers, no 
"uninterpretable" markers could possibly exist. 

3 •The K-F model fails to take into account an 



infinity of possible ramifications (or 
connotations). Bachelor frequently connotes-- 
"dissolute young man" or "charming young man" or 
"fellow on the make" or "homosexual" or "sexual 
dysfunction" or "woman hater" or "dedicated to his 
profession", etc. (e.g.. Garrison Keillor's 
Norwegian bachelor farmers). 

4. This the model refuses to consider settings or 
contexts, how a given lexical item can be read one 
way in one setting, another in another, or even in 
more than one way at a time (metaphor) • 

5. The model is limited to the verbal and 
categormatic . The concept of bachelor immediately 
suggests to me a visual image of my late uncle who 
lived with my family throughout my' childhood . 

Attempts to revise models such as K-F to accommodate these 
deficiencieL seem strained, forced and unsuccessful since the 
continuing assumption is that knowledge exists somehow 
independent of the human knowers and the cultural context within 
which they find themselves. 

Eco's proposed alternative to the K-F model will be 
unsatisfying to many of you because it acknowledges the 
impossibility of identifying a finite set of representational 
symbols. Instead it is proposed that representation is a net or 
labrynith of signs or interpretants of signs, an inconceivable 
glcbality. A concrete image may help to communicate this notion. 
Eco uses a vegetable metaphor of the rhizome — a tangle of bulbs 
and tubers appearing like rats squirming one on top of the other. 
Rhizomatic structure may be characterized as follows: 

1. Every point of the rhizome can and must be 
connected with every other point, raising the 
possibility of an infinite juxtaposition. 

2. There are no fixed points or positions in a 
rhizome, only connections (relationships). 

3. A rhizome can be broken off at any point and be 
reconnected following one of its own lines (or 
relationships ) . 

4. The rhizome is antigenealogical (which I take to 
mean is not necessarily hierarchical, and that it 
is impossible to trace anv meaning back to some 
"kernel") . 

5 .The rhizome whole has no outside or inside . 

6. A rhizome is a open chart which can be connected 
with something else in all of its dimensions; it is 
dismountable , reversible and susceptible to 
continual modification. 

7. A network of trees which open in every direction 
can create a rhizome (which is equivalent to saying 
that a network of partial trees can be 
"artificially" extracted from every rhizome (see 
Figure 5). 

8. Rhizomes can not be statically pictured without 
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changing their dynamic r temporal character 
(uncertainty principle). 
9. Since tho structure as a whole can not be 
conceptualized by either the scientist or the 
user, we are left to "local" experiences. In 
essence, we are feeling/groping our way along a 
labyrinth in the same way that Brother William does 
in The Name of the Rose (Eco, 1983). 
A model such as this acknowledges the possibility of 
unlimited semiosis but at the same time allows us to understand 
the means by which we do structure our world r tc allow us to 
constrain the potential meanings in particularized ways — through 
the operation of culture. Knowledge, then, is not "out there" 
waiting to be discovered. The world, as we know it, is culturally 
coded. What we experience as reality is really prior cultural 
codings, prior structures which have been invented by our 
culture. Eco goes on to detail a brilliant theory of sign 
production which is beyond our purposes here today. 

Abduction and Affordance 

A weakness (or ax; least a frustration) I have heard 
expressed about semiotic models of cognition is that they 
eventually lead to solipsism; that is, these models seem to be 
implying that organisms can know nothing that they themselves 
have not created or experienced. While both Deely and Eco propose 
an interaction between the physical world and the cognizing 
organism via signs, little is said about the nature of the 
elements that are interacting. Although we can not know the 
physical world directly, it surely exists and as such has 
features which constrain the sign processes. As I dash across 
Canal Street, I wil 1 endeavor to avoid the onrushing traffic in 
the belief that the cars are real and not the product of my own 
unique semiosis. Given the relatively consistent nature of sign 
structures which humans within a particular context develop, what 
does this suggest about the nature of the physical world? 

Deely' s model is fundamentally a sensation based model. The 
physical world is presented to us as bundles of features which we 
some how synthesize into a perceptual object - a process which 
sounds a lot like traditional encoding models of perception where 
impoverished stimulus information is somehow operated on by 
information processing mechanisms until the information is 
reorganized into a form consistent with our existing knowledge. 
But if these features can be combined and recombined in an 
infinite number of ways, what is the reason for the preeminence 
of some arrangements over others? Could it be that perception is 
a function of structures in the physical world, not simply 
decomposed features and/or constructions of the perceiving 
organism? 

This is essentially the question raised by J. J. Gibson 
(1979) in his ecological approach to visual perception ( it 
should be stressed he re that Gibson's work is limited largely to 
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visual perception and can only be analogously applied to other 
forms of perception and to other cognitive processes. I hope my 
use of the concepts does not do serious violence to his worki). 
Gibson begins with a notion all too rare in academic psychology r 
that his theory of perception must account for ordinary " 
perception r that of organisms moving about in their environment , 
seeing the ordinary things of daily life. Why^ he asks, would it 
be the case that if perception was simply a construction from 
sensed features { and impoverished ones at that) that our 
perceptions seem to agree so well with the environment through 
which a multitude of observers move? 

Gibson rejects the sensation based theories of perception 
which regard the perceiver as a passive receiver of impoverished 
stimulus energy on the basis of which the perceiver transforms a 
retinal image into a perception. His ecological optics abandons 
the sensation-perception distinction and proposes instead an 
ecological model of an active perceiver confronting an 
information rich stimulus environment. Gibson's notion of 
environment is very compatible with the idea of the Umwelt. For 
Gibson r an environment is that which organisms perceive, not the 
physical world which a physicist might describe. The term refers 
to the "surroundings" of an organism on a scale appropriate for 
terrestrial animals (i. e. in terms of tii;rrainsr objects , and 
events which are appropriate for organisms on this planet - sizes 
between inches and meters, times between seconds and hours, 
etc.). For Gibson, the words organism and e nvironment are an 
inseparable pair.... each implies the other. One can not talk of 
an environment in general but only of an environment with respect 
to a particular anim?l. 

The terrestrial environment, unlike the physical 
environment, consists of a medium, substances and surfaces that 
separate the medium from substances. The medium for humans is the 
gaseous atmosphere, the "air" which permits unimpeded locomotion 
from place to place, seeing, smelling and hearing of substances. 
In our world, the medium has an absolute axis of reference, the 
vertical axis defined by gravity. Substances are the "things " of 
the world, the objects or "furniture" which occupy the 
terrestrial surface. Unlike the medium, substances do not permit 
locomotion o/ transmit light. Substances are heterogeneous 
whereas the medium is relatively homogeneous. Surfaces separate 
the medium from substances. It is at the level of surface where 
all of the action in visual perception takes place. We do not 
perceive the medium or substances but only surfaces where the 
medium and substances meet. A surface is said to have a layout 
(form), texture, the property of being lighted or shaded and the 
property of a certain fraction of the illumination falling on it. 

To make a very long story much too short, visual perception 
arises when structured information froia surfaces is perceived. 
The ambient optical array (this structured information available 
in light) is described by Gibson as visual solid angles with a 
common apex at the point of observation. They are angles of 
intercept which change as the observer moves or the surface{s) 
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under observation move. But other aspects of the array do not 
change (e.g. the layout and reflectance). The perceptual system 
monitors those things that change and those things that persist 
and from this information perception , is developed. Perception is 
thus a process which develops from the interaction of an active 
perceiver in an informational ly rich environment which is 
constantly in flux . 

Of primary importance for our purposes here is Gibson's 
theory of affordances. As noted above r Gibson considers the 
environment to be the surfaces that separate substances from the 
medium in which animals live. But environments also afford things 
(such as shelter^ locomotion, etc.). There is information in 
light for perception but also for the perception of what surfaces 
afford. To perceive something is to perceive what it affords , its 
value or meaning. To quote Gibson ''The affordances of the 
environment are what it offers the animal, what it provides or 
furnishes, either for good or ill" (p. 127). But affordances do 
not exist independent of an animal; the term refers to both the 
environment and animal. "An affordance is neither an objective 
property nor a subjective property; or it is both if you like... 
(it) points both ways, to the environment and the observer" (p. 
129). The terrestrial surface, for example may be horizontal, 
flat, extended and rigid, thus affording support to certain 
terrestrial animals. But this affordance is relative to 
particular terrestrial animals, not an abstract property of the 
physical world. 

The process of perceiving affordances is called "information 
pick-up", unfortunately the least developed aspect of Gibson's 
theory. Affordances are invariants available in the ambient optic 
array and perception of affordances results from monitoring those 
aspects of the ambient optic array which persist and those which 
change. Note that t".his conception places the affordance in the 
light, not in the needs or motives of the observer. The potential 
affordance of a paper clip as a replacement for fishhook is 
available whether or not it is perceived by a particular organism 
to which the affordance is relevant. 

In my current thinking, the concept of affordance is very 
relevant to semiotic models of cognition such as those described 
above. In Deely's model, for example, abduction is a mode of 
inferencing whereby organisms attempt to make sense of the world 
by creating sign structures. In essence this process has been 
likened by Shank (in press) to "reading" the environment. But is 
our reading free to take any possible form? Can the Umwelt we 
create be entirely independent of those aspects of the 
environment relevant to us as a species or as an individual? A 
fruitful area of research for semioticians will be to investigate 
the possibilities of af fordance-1 ike constancies in our worlds. I 
fully appreciate the fact that I am making a hugh leap from a 
model of visual perception to one of cognition, especially a 
model of cognition with the scope of semiotics. I personally 
believe the leap is justified. 

To close, let me give one brief example of the sort of 
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analysis I am proposing. Take a typical secondary classroom. Are 
there af f ordance-like constancies in this situation that can be 
read? Clearly the readings can be numerous and relevant to a wide 
variety of perspectives and contexts. We can analyze the 
ideational character of the classroom discourse (as Jay Lempke 
(in press) does), observe the social interactions among the 
students, chart the patterns of student and teacher questions and 
answers, aud so on and so forth. To what extent do the particular 
events and circumstances we observe lead us to the 
identification of this as a classroom, a pedagogical technique of 
a certain type. What is essentxal (persistent) and what is 
changing (variable). Do certain metaphors seem to account for our 
observations better than others? 

The underlying motivation of such research is that over time 
it will eventually lead to ever more adequate conceptions of the 
affordances available in this stimulus information, ^ike Peirce, 
I believe that our inquiry will eventually l^iad us closer to the 
dynamic object, to an understanding of the world as it is, 
unmediated by signs. But since this quest is of the nature of all 
cognition, why should our inquiry be any different? 



^ Paper presented at the annual meeting of the American 
Educational Research Association, New Orleans, April, 1988. 
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BEST COPY AVAILABLE 



bachelor 

I 

Noun 



(Hunwn) (Animal)^^^ 

[Having the academic (^faIe) 
degree conferred for 
(Adult) (Young) completing the first 

four years of college] 



(Never-Married) (Knight) 
<Jx> 

[Serving under ths 
standard of another] 



<w,> 



\ 

(Young) 
(sLl) 

[When without a 
mate during the 
breeding time] 




